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DETAILED ACTION 
Election/Restrictions 

1. Restriction to one of the following inventions is required under 35 U.S.C. 121 : 

I. Claims 1-43, drawn to a system for tracking at least one object in a scene using a 
model based on a state estimate of probabilistic configurations and color 
observations classified in class 382, subclass 103 (target tracking or detecting). 

II. Claims 44-47, drawn to a method for automatically learning a color-based model 
based on the use of Dirichlet functions classified in class 382, subclass 155 
(learning systems). 

The inventions are distinct, each from the other because of the following reasons: 

2. Inventions I and II are related as combination and subcombination. Inventions in this 
relationship are distinct if it can be shown that (1) the combination as claimed does not require 
the particulars of the subcombination as claimed for patentability, and (2) that the 
subcombination has utility by itself or in other combinations (MPEP § 806.05(c)). In the instant 
case, the combination as claimed does not require the particulars of the subcombination as 
claimed because claim 1, the broadest combination claim, does not require the use of a Dirichlet 
function in order to generate a state estimate or make color observations as required by claim 44, 
the broadest subcombination claim. The subcombination has separate utility. The method 
proposed by claim 44, the subcombination claim, can be used in any image processing 
environment where the use of a model is necessary such as template matching, inspection of 
manufactured products, or pattern recognition. 
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3. Because these inventions are distinct for the reasons given above and have acquired a 
separate status in the art because of their recognized divergent subject matter, restriction for 
examination purposes as indicated is proper. 

4. During a telephone conversation with Mark A. Watson (Reg. No. 41,370) on June 3, 
2003 a provisional election was made without traverse to prosecute the invention of Group I, 
claims 1-43. Affirmation of this election must be made by applicant in replying to this Office 
action. Claims 44-47 withdrawn from further consideration by the examiner, 37 CFR 1.142(b), 
as being drawn to a non-elected invention. 

Claim Objections 

5. The following quotation of 37 CFR § 1 .75(a) is the basis of objection: 

(a) The specification must conclude with a claim particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention or discovery. 

6. Claims 24-27 are objected to under 37 CFR § 1.75(a) as failing to particularly point out 
and distinctly claim the subject matter which the applicant regards as his invention or discovery. 

Claim 24 recites the limitation "second histogram" in line 2. There is insufficient 
antecedent basis for this limitation in the claim. 

Claim 25 recites the limitation "the third histogram 55 in line 2. There is insufficient 
antecedent basis for this limitation in the claim. 

The examiner assumes that claim 24 depends from claim 23 for examination purposes. 

Claim Rejections - 35 USC § 102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 
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(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

8. Claims 1-8, 10, 13-22, 28, 29, and 33-35 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Birchfield (the article titled "Elliptical Head Tracking Using Intensity Gradients 
and Color Histograms"). 

As applied to claim 1, which is representative of claims 15 and 16, Birchfield discloses a 
system for tracking at least one object in at least one sequential image, comprising: a general 
purpose computing device; and a computer program comprising program modules executable by 
the computing device, wherein the computing device is directed by the program modules of the 
computer program (see section 5.5: The reference describes that a 200 MHz Pentium Pro 
microprocessor was used.) to: (a) generate a state estimate defining probabilistic configurations 
of each object for each sequential image (see section 2: "head's state, i.e. The reference describes 
estimating a position and size of the head (i.e. an object) in an image (i.e. generate a state 
estimate) using a vertical ellipse with a fixed aspect ratio of 1.2 (i.e. defining probabilistic 
configurations of each object for each sequential image).); (b) generate observations of pixel 
color for each sequential image (see section 4: The reference describes determining a model 
histogram by counting all of the pixels inside the ellipse and an image histogram.); (c) 
automatically learn a color-based object model using the state estimate and the observations (see 
section 2: The reference describes the use of the combination of the intensity gradients around 
the object's boundary (i.e. state estimate) and the color histogram of the object's interior (i.e. 
observations) to form a model for tracking the object.); and (d) automatically track each object 
using the learned color-based model with a color-based tracking function (see section 5.3: The 
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reference describes that information obtained from the color module and the gradient module 
(i.e. a color-based model) are used to obtain ideal tracking results.). 

As applied to claim 2, Birchfield discloses that generating the state estimate comprises 
determining the probabilistic configurations of each object using an initial image processing 
program module (see section 5.5: The reference describes the use of a 200 MHz Pentium Pro 
microprocessor. This computer has all of the program modules including an initial image 
processing unit). 

As applied to claim 3, Birchfield discloses that the initial image processing program 
module employs a tracking system comprising a tracking function in combination with an object 
model for probabilistically detecting object configuration information (see section 2: The 
reference describes the use of a tracking function (see equation (1)) that makes use of the 
combination of the intensity gradients around the object's boundary and the color histogram of 
the object's interior (i.e. an object model) for detecting the object configuration.). 

As applied to claim 4, Birchfield discloses that the initial image processing program 
module employs a contour-based tracking function in combination with a contour-based object 
model for probabilistically detecting object configuration information (see section 3: The 
reference describes that the gradient module performs much like a contour-based tracker.). 

As applied to claim 5, which is representative of claim 19, Birchfield discloses that 
generating the observations of pixel color comprises collecting pixel color information over the 
entirety of each image (see section 4: The reference describes determining a histogram of the 
entire image (i.e. collecting pixel color information over the entirety of each image).). 
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As applied to claim 6, Birchfield discloses that generating the observations of pixel color 
comprises collecting pixel color information over specific portions of each image (see section 4: 
The reference describes determining a model histogram by counting the pixels inside the ellipse 
(i.e. collecting pixel color information over specific portions of each image).). 

As applied to claim 7, Birchfield discloses generating the observations of pixel color 
employs the state estimate to identify specific relevant regions of each image over which pixel 
color information will be collected (see section 4: The reference describes determining a model 
histogram by counting the pixels inside the ellipse, where the ellipse is the original state estimate 
as described in section 2 of the reference.). 

As applied to claim 8, which is representative of claim 21, Birchfield discloses that 
generating the observations of pixel color comprises automatically generating a first probability 
distribution function modeled using a first histogram to represent a range of observed pixel 
colors (see section 4: The reference describes the use of a histogram for generating color 
observations of the object. A histogram, as described in the claim, is a probability distribution 
function.). 

As applied to claim 10, which is representative of claim 22, Birchfield discloses program 
module for automatically learning the color-based object model automatically computes a second 
probability distribution function modeled using a second histogram to represent a background for 
each image (see section 4: The reference describes forming an image histogram, which is a 
histogram of the entire image minus the model (i.e. a background image). Also, as described 
above, a histogram is a probability distribution function.). 
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As applied to claim 13, Birchfield discloses that automatically learning the color-based 
object model comprises performing a bin-by-bin comparison between the first histogram and the 
second histogram (see section 4: The equation for determining O c performs a bin-by-bin 
comparison between the model histogram (i.e. the first histogram) and the image histogram (i.e. 
the second histogram).). 

As applied to claim 14, Birchfield discloses that bins in the first histogram having values 
exceeding corresponding bins in the second histogram correspond to those color ranges 
representing the learned color-based object model (see section 4: The reference describes that the 
equation for determining <D C is used, in part, to represent the color-based object model.). 

As applied to claim 17, Birchfield discloses that a confidence measure is associated with 
the observations of pixel color (see section 4: The reference describes that an offline, as well as 
an online, histogram is obtained. The offline histogram is used as a confidence measure that the 
online histogram (i.e. observations of pixel color) is actually a histogram representation of the 
head.). 

As applied to claim 18, Birchfield discloses that the observations of pixel color are 
weighted in proportion to the confidence measure (see section 4: The equation <X> C is used to 
weight the online histogram (i.e. observations of pixel color) by the values of the offline 
histogram (i.e. the confidence measure).). 

As applied to claim 20, Birchfield discloses that the observations of pixel color are 
collected over specific portions of each image wherein the state estimate has a probability greater 
than a minimum threshold level (see section 4: The reference describes determining a model 
histogram by counting the pixels inside the ellipse (i.e. collecting pixel color information over 
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specific portions of each image). The ellipse corresponds to the state estimate. Therefore, an 
ellipse will only be formed if the state estimate exists, so the state estimate has to have a 
probability greater than zero (i.e. a minimum threshold level).). 

As applied to claim 28, Birchfield discloses that generating the state estimate comprises 
processing each image with an initial object model and an initial tracking function (see section 2: 
The reference describes modeling the head as an ellipse (i.e. initial object model) and the use of 
equation (1) (i.e. an initial tracking function).). 

As applied to claim 29, Birchfield discloses that the initial object model is iteratively 
replaced with the learned color-based object model and the initial tracking function is replaced 
with a color-based tracking function to improve the accuracy of the learned color-based object 
model (see section 2: Equation (1) is constantly updated based on the values obtained from the 
gradient module and the color module.). 

As applied to claim 33, Birchfield discloses a process for gathering the sequential images 
(see section 1 : The reference describes that the system can be implemented in various 
applications such as video conferencing, where a video camera would be used to gather the 
sequential images.). 

As applied to claim 34, Birchfield discloses a computer-readable memory for identifying 
the configuration of objects of interest in a scene, comprising: a computer-readable storage 
medium; and a computer program comprising program modules stored in the storage medium, 
wherein the storage medium is so configured by the computer program that it causes the 
computer to: generate an initial configuration estimate for objects of interest within the scene 
(see section 2: The reference describes estimating a position and size of the head (i.e. an object) 
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in an image (i.e. generate an initial configuration estimate) using a vertical ellipse.); identify 
pixel color information within the scene that is relevant to a learned color-based object model 
(see section 4: The reference describes determining a model histogram by counting all of the 
pixels inside the ellipse and an image histogram.); automatically learn the color-based object 
model by determining probabilistic relationships between the initial configuration estimates and 
the pixel color information (see section 2: The reference describes the use of the combination of 
the intensity gradients around the object's boundary (i.e. initial configuration estimates) and the 
color histogram of the object's interior (i.e. pixel color information) to form a model. This 
combination is a probabilistic relationship since the area where the pixel color information is 
obtained from is based on the initial configuration estimates.); and generate a final configuration 
estimate for objects of interest in the scene by using the color-based object model in combination 
with a color-based tracking function (see section 2: Equation (1) is used to determine a final 
configuration estimate for the objects in the scene.). 

As applied to claim 35, Birchfield discloses that the program module for generating the 
initial configuration estimate further includes an initial object model and an initial tracking 
function, and wherein the initial object model is comprised of parameters used by the initial 
tracking function for determining the configuration of objects within the scene (see section 2: 
The reference describes modeling the head as an ellipse (i.e. initial object model) and the use of 
equation (1) (i.e. an initial tracking function).). 

Claim Rejections - 35 USC §103 
9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 



Application/Control Number: 09/592,750 Page 10 

Art Unit: 2621 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

10. Claims 9, 36, and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Birchfield (the article titled "Elliptical Head Tracking Using Intensity Gradients 
and Color Histograms") and Koller et al. (the article titled "Using learning for approximation in 
stochastic processes"). The arguments as to the relevance of Birchfield in the rejections of claims 
1, 8, 34, and 35 are incorporated herein. 

As applied to claims 36 and 37, Birchfield discloses representing the pixel color 
information and the background image information using a probability distribution function (see 
section 4: The reference describes using a histogram (i.e. a probability distribution function) to 
represent the model (i.e. pixel color information) and the image (i.e. background information). 

Claim 9, as well as claims 36 and 37, call for the use of a Dirichlet function. Birchfield 
does not teach the use of a Dirichlet function; however, Koller et al., in the same field of 
endeavor of image processing, and the same problem solving area of object tracking, describes 
the use of such a function (see column 10: The reference describes the use of a Dirichlet prior for 
use with a multinomial distribution.). 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to modify Birchfield by adding the use of a Dirichlet function as taught by 
Koller et al. because the use of a Dirichlet function "serves to 'spread out' some of the 
probability mass over unobserved states, increasing the amount of exploration done for 
unfamiliar regions of the space" (see Koller et al: column 11). 
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Allowable Subject Matter 



11. Claims 11, 12, 23, 30-32, and 38-43 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

12. Claims 24-27 are also objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten to correct for the 37 CFR § 1 .75(a) objection described above 
and in independent form including all of the limitations of the base claim and any intervening 
claims. 



13. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Marks et al. (U.S. Patent No. 5,845,009 A) is pertinent in that the reference teaches a 
method of identifying and tracking an object in an image using statistical models based on color 
and shape. 

Toyama et al. (U.S. Patent No. 6,502,082 Bl) is pertinent in that the reference teaches the 
use of a statistical model in tracking an object based on various tracking modalities. 

Cosatto et al. (U.S. Patent No. 5,864,630 A) is pertinent in that the reference discloses a 
tracking system that uses color and shape information. 

Darrell et al. (U.S. Patent No. 6,445,810 B2) is pertinent in that the reference discloses a 
method for tracking an object using color segmentation and pattern classification. 



Conclusion 
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14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J. Miller whose telephone number is (703) 306-4142. The 
examiner can normally be reached on M-F 8:00-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo Boudreau can be reached on (703) 305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-93 14. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-4750. 



Ryan J. Miller 
Examiner 
Art Unit 2621 



Ryan J. Miller 
June 13,2003 
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